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The development of information and communication technology (ICT) during 
the era of the fourth industrial revolution, the impact of the COVID-19 
pandemic in 2020, and the government’s call for large-scale social restrictions 
have led to the emergence of online learning systems (OLS) in higher 
education. This study develops a measurement model for the success of OLS 
based on the DeLone & McLean model. Surveys were conducted on a sample 
of 175 students from domestic and international universities. Data processing 
used the partial least squares structural equation modeling or PLS-SEM 
method, and root cause analysis. The results show that platform quality has a 
positive influence on OLS success, whether mediated by user satisfaction or 
OLS usage. Social influence has a positive effect on OLS success, mediated 
by OLS usage. User computer anxiety has a negative effect on OLS success, 
mediated by user satisfaction. Recommendations to improve OLS success 
include adjusting internet package rates to make them more affordable, 


improving signal strength to various locations for better coverage, limiting the 
number of users in one learning session for more effective OLS, and the need 
for organizational support in using the right applications. 
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1. INTRODUCTION 

Ever since the beginning of the COVID-19 outbreak, the Indonesian authorities have enforced 
regulations to limit gatherings in public spaces, businesses, and educational institutions. As a consequence, 
students and educators are required to conduct all their academic activities from their homes by utilizing online 
platforms for virtual meetings and classes [1]-[5]. Online learning systems (OLS), which are defined as various 
functions to support virtual classrooms to improve the quality of teaching and learning activities [6]-[8], are a 
great opportunity for application and internet service providers to collaborate in gaining profits [1], [9]. 

With the work-from-home (WFH) program, requests for an internet network installation at home 
increased dramatically in March 2020 compared to the previous month by 30-40% [10]. At the time of 
COVID-19, the use of educational e-learning platforms with video conferencing tools dominated teaching and 
learning, namely Zoom, Microsoft Teams, Moodle, and Google Classroom and their effectiveness was proven 
to offer the right solution for emergency online teaching difficulties [11]. Zoom usage per day could reach 200 
million in March, and 300 million in the following month compared to December 2019 of 10 million and the 
initial launch of Zoom in 2013 of 3 million usages [12], [13]. Meanwhile, the use of Google Meet per day 
reached 100 million, which increased by 300% since January [14]. 
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People are using Zoom because it is easy to use and free of cost, but its competitors are catching up. 
Measuring video conferencing application performance metrics against the performance of the three most 
popular platforms in education and business, namely Google Meet, Microsoft Teams, and Zoom, it was found 
that there are substantial differences in the way the three applications treat video and audio streaming. While 
each platform has its benefits, there are no ideal applications, their performance depending on audio, video, 
or network bandwidth [15]. Even though the OLS can be a solution for learning activities when the offline 
system cannot be implemented, it does not mean that there are no negative impacts. According to Sabeh et al. 
[16], 83.3% of respondents agree that students’ motivation and seriousness in learning decrease when 
undergoing OLS, and 60.8% of respondents agree that OLS can cause students to become addicted or excessive 
in using gadgets. 

There are several methods for determining the efficacy of an information system (IS). The usage of 
numerous models is one of the most widely used and dependable approaches to evaluating IS. Among them, 
the DeLone and McLean model (D&M model) is a well-known and widely used tool [17], [18]. This model 
was first introduced in 1992 and was upgraded with major revisions in 2003. The D&M model is intended to 
assess many elements of an IS, including quality, user happiness, system usage or intention to use, and net 
benefits [8], [19]—[23]. 

By modifying the paradigm, the D&M model is still frequently utilized as study material today. 
Among these are study findings from [6], [12], [21], [24], [25], indicating that system quality, information 
quality, and service quality all have a beneficial impact on user satisfaction with OLS. Furthermore, according 
to Aldholay et al. [20] study findings, system quality, information quality, and service quality all have a 
beneficial influence on behavioral intention to use. According to research findings from [6], [21], [24], [25], 
user happiness has a favorable influence on net benefits. 

Nonetheless, research done by Hiremath et al. [26] and Jeyaraj [27] show that the quality of 
information has no effect on user satisfaction. Similarly, the outcomes of Ouajdouni et al. [28] research indicate 
that the quality of the system has little effect on OLS use. According to Shahzad et al. [12], the influence of 
system quality, information quality, and service quality on e-learning portals is indirect, mediated through 
system usage and user satisfaction as intermediary factors. 

This study uses the platform quality variable, which is a fusion of system, information, and service 
qualities [29]. As per the findings of Lee et al. [29], the quality of the platform has a favorable influence on 
platform loyalty. Additionally, according to Shahzad et al. [12] and Ouajdouni et al. [28] research suggests 
that user satisfaction plays a crucial role in the effectiveness of OLS. The research results from Ouajdouni 
et al. [28] show that social influence has a positive effect on the use of OLS, while computer user anxiety has 
a negative effect on user satisfaction in OLS. Hence, based on these observations, we can formulate: i) H1: 
Platform quality has a positive effect on the success of OLS through user satisfaction; ii) H2: Platform quality 
has a positive effect on the success of OLS through the use of OLS; iii) H3: Social influence has a positive 
effect on the success of OLS through the use of OLS; and iv) H4: User anxiety in using computers has a 
negative effect on the success of OLS through user satisfaction. 

Based on empirical phenomena, this research analyzes the influence of platform quality, social and 
user computer anxiety on the success of OLS using video conferencing applications both through user 
satisfaction and OLS use. In addition, this research analyzes the root causes and solutions to the not yet optimal 
success of this OLS. This research is expected to provide practical benefits and theoretical contributions 
regarding measuring the quality and success of information systems. Hopefully, this research can also be used 
as a reference for optimizing OLS based on/using video conferencing applications, especially in improving the 
various dimensions that influence it. This research is different from previous studies for promoting a novelty 
i.e., combining variables in the research of authors [19], [29] (platform quality), and [28] (social influence and 
user computer anxiety). Based on the literature review, previous research, and hypotheses, the theoretical 
framework of this research can be illustrated in Figure 1. 
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Figure 1 Theoretical framework 
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2. RESEARCH METHOD 

The research method used is a mixed method, an intellectual and practical synthesis based on 
qualitative and quantitative research [30]. This research is also associative, which asks about the relationship 
between two or more variables that will function to explain, predict, and control phenomena [22]. In the 
research context, the researchers conduct a verification analysis, testing the research hypothesis (quantitative): 
the influence of platform quality, and social and user anxiety on the success of OLS through the use and 
satisfaction of OLS users. Once the survey data has been collected, the researchers conduct a verification 
analysis: testing the research hypotheses that have been proposed by the authors (quantitative) and looking for 
light and heavy obstacles to find solutions to optimize the success of OLS using video conferencing 
applications (qualitative). This study encompasses three categories of latent factors (independent variables, 
mediating variables, and dependent variables). Independent variables include the social impact and learners’ 
computer anxiety, mediating variables are OLS usage and user satisfaction, and the dependent variable is OLS 
performance. Variable operationalization is described in Table 1. The measures in this study are taken on 
a 5-point ordinal/Likert scale (1. strongly disagree, 2. strongly disagree, 3. strongly disagree, 4. Agree, and 5. 
strongly agree). 


Table 1. Variable operationalization 
Latent variable Manifest variables (indicators) 
Platform quality [6], [25] Complete system information 
System information is easy to understand 
Accurate system information 
Information systems meet the needs 
Interesting system display 
System problems are quickly serviced 
Social influence [28] Important person 
People who influence behavior 
People whose opinion is valued 
Organization 
User computer anxiety [28] Nervousness 
Bad premonition 
Inconveniences 
OLS use [28] Obtain information 
Publish information 
Communicate with peers and educators 
Do coursework 
User satisfaction [6], [25] Satisfied with use 
Satisfied with system information 
Satisfied with system performance 
Self-confident 
Satisfying needs 
Overall interaction 
OLS success [6], [25], [28] Positive impact 
Opportunity for more achievement 
Facilitate knowledge transfer 
Help increase independence 
OLS performance was good overall 
OLS was successful overall 


The population used as respondents in this study are Indonesian students currently studying at 
domestic and foreign universities. According to Hair et al. [31], the optimal number of samples is 5-10 times 
the highest indicator of one latent variable or 10 times the highest number of structural paths leading to a latent 
variable. The researchers determined 175 samples of Indonesian students currently studying at local and foreign 
universities. The sample of respondents currently studying at domestic universities are 28 public and private 
universities. Their domiciles come from several provinces in Indonesia, including North Sumatra, DKI Jakarta, 
West Java, Central Java, East Java, South Sumatra, Aceh, D.I. Yogyakarta, Riau Archipelago, East Nusa 
Tenggara, Riau, and West Sumatra. Samples of Indonesian student respondents from foreign universities came 
from Australia (University of Melbourne, Australian National University, Monash University, University of 
Queensland, and the University of Sydney), China (Suzhou University of Science and Technology and Xi’an 
Jiaotong Liverpool University), and Singapore (Curtin University Singapore). 

Structural model testing, which aims to test the hypothesis, uses partial least square (PLS) analysis 
which is a variance-based structural equation modeling (SEM) method [13], [32]. The evaluation of this 
research model is divided into two stages, the evaluation measurement (outer model) and testing of the 
structural model (inner model). After knowing the results of hypothesis testing, a root cause analysis is carried 
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out. The 5-whys is one of the popular analytical methods that aims to find solutions that start from the root of 
the problem [33]. The measurement (outer model) evaluation consists of convergent validity, discriminant 
validity, and reliability in a construct/variable that is built, which must be able to meet the applicable parameters 
as seen in Table 2. According to the table, all indicators that are part of the latent variable have an outer loading 
factor value greater than 0.7, which is higher than other variables’ cross-loading. This indicates that the 
requirements for both convergent and discriminant validity have been met. 

Table 3 shows all average variance extracted or AVE values are above 0.5. Table 4 shows the Fornell- 
Larcker criterion for assessing discriminant validity involving latent variables to prevent multicollinearity 
problems. The table display the VAVE value of the user satisfaction or LS correlation variable, is 0.866, which 
is greater than the correlation value of the LS variable with user computer anxiety or LCA (-0.229) and others. 
This also applies to other variables, where the value of VAVE on the variable itself shows a greater value than 
the correlation between latent variables. Table 5 shows all reliability parameters have a value above 0,7, 
meaning they have met the reliability requirements. Thus, the evaluation measurement has fulfilled all the 
requirements, and it can be said that the model that has been formed is valid and reliable. 


Table 2. Test parameters of validity and reliability [34], [35] 


Outer model Parameter Requirements 
Convergent validity Outer loading >0.7 
Average variance extracted (AVE) >0.5 
Discriminant validity Fornell-Larcker criterion VAVE > correlation between latent variables 
Cross loading Loading factor > correlation between latent variables 
and manifest 
Reliability Composite reliability, Cronbach’s alpha and rho_A >0.7 


Table 3. Average variance extracted (AVE) 


Variable AVE 
User computer anxiety 0.770 
User satisfaction 0.749 
OLS success 0.741 
OLS use 0.655 
Platform quality 0.631 
Social influence 0.661 


Source: The results of data processing by PLS-SEM 


Table 4. Fornell-Larcker criterion for assessing discriminant validity 


Variable LCA LS OlsSce  OlsU PQ ScInf 
User computer anxiety (LCA) 0.877 
User satisfaction (LS) -0.229 0.866 
Online learning system success (OlsSc) -0.296 0.814 0.861 
Online learning systen use (OlsU) -0.371 0.645 0.669 0.810 
Platform quality (PQ) -0.127 0.631 0.538 0.528 0.794 
Social influence (ScInf) -0.168 0.464 0.425 0.460 0.535 0.813 


Source: The results of data processing by PLS-SEM 


Table 5. Reliability parameter test results 


Variable Cronbach’s alpha _ _rho_A Composite reliability 
User computer anxiety 0.855 0.902 0.909 
User satisfaction 0.933 0.934 0.947 
OLS success 0.930 0.933 0.945 
OLS use 0.827 0.833 0.884 
Platform quality 0.883 0.887 0.911 
Social influence 0.830 0.843 0.886 


Source: The results of data processing by PLS-SEM 


Table 6 illustrates the impact of platform quality and social influence on OLS usage, resulting in a 
coefficient of determination of 0.323. This means that 32.3% of OLS usage can be accounted for by the 
influence of platform quality and social influence, whereas the remaining 67.7% is attributable to other 
variables beyond the scope of this study. Furthermore, the effect of platform quality and user computer anxiety 
on user satisfaction yields a coefficient of determination of 0.421. This implies that 42.1% of user satisfaction 
can be attributed to the influence of platform quality and user computer anxiety, while the remaining 57.9% 
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can be explained by other variables not examined in this study. Finally, the impact of platform quality, social 
influence, user computer anxiety, OLS usage, and user satisfaction on OLS success results in a coefficient of 
determination of 0.701. This indicates that 70.1% of OLS success can be explained by the influence of platform 
quality, social influence, user computer anxiety, OLS usage, and user satisfaction, whereas the remaining 
29.9% is attributable to other variables not considered in this study. Figure 2 shows the result of testing the 
outer and inner models using the Smart PLS software version 3. In this case, the researchers use a significance 
level of 5%. 


Table 6. The coefficient of determination in structural model testing 


Variable R Square __R square adjusted 
OLS Use 0.323 0.315 
Learner satisfaction 0.421 0.414 
OLS Success 0.701 0.692 


Source: The results of data processing by PLS-SEM 
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Figure 2. Outer and inner model test results 


3. RESULTS AND DISCUSSION 

Based on the processing of the respondent’s profile, most respondents are female, coming from 
Gen Z and currently occupying the bachelor/S1 level. The intensity of using video conferencing applications 
is an average of 4-6 times a week. The most popular video conferencing application is Zoom. Table 7 shows 
all the direct effect path coefficient values in the structural model test with significant results (P values <0.05 
or T statistics >T table 1.96) and have a positive or negative direction. Table 8 shows all the path coefficient 
values of the indirect effect on the structural model testing; the results are significant and have a positive or 
negative direction. The outcomes of the study are presented in Table 9, indicating that: i) The platform’s quality 
has a favorable impact on OLS success through user satisfaction; ii) Platform quality has a constructive impact 
on OLS success through OLS usage; iii) Social influence has a beneficial impact on OLS success through OLS 
usage; and iv) User computer anxiety has an adverse effect on OLS success through user satisfaction. 


Table 7. Direct effect path coefficient values in structural model testing 


Path analysis Original sample Sample mean Standard deviation ___T stat__P values 
User computer anxiety — User satisfaction -0.151 -0.151 0.062 2.439 0.008 
User satisfaction — OLS success 0.655 0.651 0.064 10.207 0.000 
OLS use — OLS success 0.221 0.222 0.073 3.041 0.001 
Platform quality — User satisfaction 0.612 0.615 0.053 11.622 0.000 
Platform quality — OLS use 0.395 0.393 0.076 5.226 0.000 
Social influence — OLS use 0.249 0.260 0.088 2.842 0.002 


Source: The results of data processing by PLS-SEM 
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Table 8. Indirect effect path coefficient values in structural model testing 


Path analysis Original sample Sample mean Standard deviation T stat P values 
Platform quality —> OLS use — OLS success 0.087 0.087 0.033 2.610 0.005 
User computer anxiety — user satisfaction > 
OLS success -0.099 -0.097 0.038 2.622 0.005 
Social influence — OLS use — OLS success 0.055 0.058 0.027 2.018 0.022 
Platform quality — user satisfaction — OLS 0.401 0.401 0.056 7.110 0.000 


success 
Source: The results of data processing by PLS-SEM 


Table 9. Hypothesis test results 


Hypothesis Result Proof Conclusion 

Hl Accepted T stat=7.11>1.96 Platform quality has a positive effect on the success of OLS through 
P value=0.000<0.05 (significant) user satisfaction 

H2 Accepted T stat=2.61>1.96 Platform quality has a positive effect on the success of OLS through 
P value=0.005<0.05 (significant) the use of OLS 

H3 Accepted T stat=2.02> 1.96 Social influence has a positive effect on the success of OLS through 
P value=0.022<0.05 (significant) the use of OLS 

H4 Accepted T stat=2.62> 1.96 User anxiety in using computers has a negative effect on the success 


P value=0.005<0.05 (significant) of OLS through user satisfaction 


3.1. The effect of platform quality on OLS through user satisfaction 

The effect of platform quality on OLS through user satisfaction is significant and positive. It can be 
interpreted that the higher the platform quality, the higher OLS success through high user satisfaction. Based 
on observations, most users are satisfied with the performance of the video conferencing application system 
used (agree=41.71% and strongly agree=30.86%). This is because the platform has provided services and 
resolved problems quickly (agreed=41.14% and strongly agree=18.29%), provided adequate information 
(agree=39.43% and strongly agree=41.71%) and regularly and periodically release/update the latest system. 
The results of testing the path of influence are in line with the basic theory of DeLone & McLean’s information 
system success model and research by researchers [12], [28]. 


3.2. The effect of platform quality on the OLS success through the OLS use 

The impact of platform quality on OLS success through the utilization of OLS is positive and 
statistically significant. This implies that OLS success increases with higher platform quality, as well as the 
use of good OLS. The majority of students were observed to use the system effectively, as indicated by the 
positive tendency of the latent variable. This is partly due to the high quality of the platform application, which 
is reflected in the positive tendency of the latent variable. These findings align with the fundamental principles 
of DeLone & McLean’s information system success model and the research conducted by Shahzad et al. [12]. 


3.3. The social influence on the variable OLS success through OLS use 

The social influence on the variable OLS success through OLS use is significant and positive. It can 
be interpreted that the higher the social influence, the more success of OLS through the use of good OLS. 
Based on observations, Zoom is the most widely used application because it is influenced by the surrounding 
environment/organization. However, most users only use the trial version, which is limited in duration. Even 
though most users are able to use the system well (the latent variable tends to be positive), the time limit reduces 
its effectiveness. In this case, when the duration limit runs out, users have to reopen the application, which 
takes time. Buying a Zoom package or using another application can be a solution. The results of testing the 
path of influence are in line with the basic theory of DeLone & McLean’s information system success model 
and research by researchers [28], [36]. 


3.4. The effect of user computer anxiety on the OLS success through the user satisfaction 

The effect of user computer anxiety on the OLS success through user satisfaction is significant and 
negative. It can be interpreted that the lower the user’s anxiety in using the computer, the more success of OLS 
through high user satisfaction. The success of OLS is also influenced by user satisfaction the only user 
confidence in using the computer. Based on observations, most users are confident using computers 
(agree=36.00% and strongly agree=26.86%. Users who are not confident will increase their anxiety in using 
computers for OLS; thus, when online learning uses conference applications and videos, they are not actively 
following. The results of testing the path of influence are in line with the basic theory of DeLone & McLean’s 
information system success model and research by previous studies [28], [37]. 

The researchers use root cause analysis with the 5-whys method to find the right solution, so it is 
expected to optimize OLS. Based on the 5-whys method, the following questions arise based on previous 
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answers (tic-tac-toe), answers based on evidence, and evidence based on real conditions in the research 
phenomenon [38]. The ‘why tree diagram’ can be seen in Figure 3. Based on ‘why tree diagram’ analysis, 
suggestions for increasing the success of OLS are adjusting internet package rates which are still quite high so 
that they are more affordable, increasing signal strength to various locations to make it more evenly distributed, 
limiting the number of users in one lesson session so that OLS is more effective, and the need for organizational 
support in using the right application. 
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Figure 3. Why tree diagram 


4. CONCLUSION 

The quality of the platform has a positive impact on the success of OLS, which is facilitated by the 
utilization of OLS and user satisfaction as intermediate variables. In addition, social influence positively impacts 
OLS effectiveness, which is also mediated by the utilization of OLS. However, the negative effect of user 
computer anxiety on the success of OLS is mediated by user satisfaction. Suggestions for increasing the success 
of OLS are adjusting internet package rates which are still quite high so that they are more affordable, increasing 
signal strength to various locations to make it more evenly distributed, limiting the number of users in one lesson 
session so that OLS is more effective, and the need for organizational support in using the right application. 

Although the research has been carried out optimally, it still has some limitations. This research is 
limited to OLS using video conferencing applications, so the results cannot be generalized to other OLS, such 
as mixed or hybrid learning models, asynchronous, and synchronous online courses in real-time. This study 
selected respondents limited to Indonesian students studying at domestic and foreign universities. Further 
research can use other respondents, for example employees, professionals/business owners, housewives, and 
entrepreneurs. This study uses the variables platform quality, social influence, computer user anxiety, OLS use, 
and user satisfaction in measuring the success of OLS. Further research may involve other variables, such as 
relationship quality, perceived usefulness, and loyalty. 
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